Unacceptable use of substandard metrics in policy decisions which mandate large reductions in animal-source foods

Many recent very influential reports, including those from the Global Burden of Disease (GBD) Risk Factor Collaborators, the EAT-Lancet Commission on Food, Planet, Health, and the Lancet Countdown on Health and Climate Change, have recommended dramatic reductions or total exclusion of animal-source foods, particularly ruminant products (red meat and dairy), from the human diet. They strongly suggest that these dietary shifts will not only benefit planetary health but also human health. However, as detailed in this perspective, there are grounds for considerable concern in regard to the quality and transparency of the input data, the validity of the assumptions, and the appropriateness of the statistical modelling, used in the calculation of the global health estimates, which underpin the claimed human health benefits. The lessor bioavailability of protein and key micronutrients from plant-source foods versus animal-source foods was not adequately recognised nor addressed in any of these reports. Furthermore, assessments of bias and certainty were either limited or absent. Despite many of these errors and limitations being publically acknowledged by the GBD and the EAT-Lancet authors, no corrections have been applied to the published papers. As a consequence, these reports continue to erroneously influence food policy decisions and international dietary guidelines, such as the World Wildlife Fund’s Livewell Diet, and the Nordic Nutrition Recommendations 2023.

health promoting substances from nutritious foods, and avoids the consumption of health-harming substances" 17 .The World Health Organisation similarly describes healthy diets as "helping to protect against malnutrition in all its forms, as well as noncommunicable diseases, including diabetes, heart disease, stroke and cancer." 18n the GBD 2019 Risk Factors Study, for both red and processed meats, the level of exposure associated with the lowest level of risk, called the theoretical minimum risk exposure level (TMREL), was, by default, set to zero.This was very puzzling, as the contribution of moderate consumption of red and processed meats to nutrient adequacy, appeared to be totally ignored.Red and processed meats are rich in all essential amino acids and in many commonly lacking micronutrients, including iron, zinc, vitamin B 12 and vitamin D 3 [19][20][21] .Whilst red and processed meats are not the only sources for these nutrients, for many populations worldwide, they are the most important bioavailable sources.If the current public health message advising moderate consumption of red and processed meats, as part of a healthy balanced diet, was to be replaced by GBD 2019 guidance that any intake of such meats was harmful, it is highly likely that the prevalence of child and maternal malnutrition, iron deficiency anaemia, and elderly sarcopenia, would be greatly increased.It is of importance that the GBD risk factor collaborators have responded to this issue, and confirmed that the TMREL for unprocessed red meat in future reports will not be zero 22,23 .Even more recently, they have stated that the 95% uncertainty interval for this estimate is very wide (0-200 g/day) 11 this indicates that the optimal intake of red meat could be as high as 200 g a day.
The EAT-Lancet planetary health reference diet does allow low quantities of red or processed meats and eggs to be consumed, and includes moderate amounts of seafood and poultry.However the diet largely consists of vegetables, fruits, whole grains, legumes, nuts and unsaturated plant oils, and only 13% of calories are from animal-source foods 3 (Fig. 1a).
In 2019, the EAT-Lancet Commission were confident that this diet would meet all nutritional requirements of all adults and of children older than 2 years.However, others questioned whether the considerable limitation of animal-source foods in the diet would negatively impact on protein and micronutrient adequacy, particularly for women, children and the elderly, and would result in adverse consequences for developing and aging brains [24][25][26] .Hence, I welcome the recent acknowledgement, by at least some of the EAT-Lancet Commissioners, that this first version of the planetary health diet would indeed result in significant essential micronutrient shortfalls 27 .This particularly pertains to micronutrients found in higher quantities, and in more bioavailable forms, in animal-source foods, such as vitamin B 12 , calcium, iron and zinc.It appears that insufficient attention was paid to the latest evidence on recommended nutrient intakes, and to the differences in micronutrient bioavailability from plant-source foods versus animal-source foods 27,28 .In order to achieve micronutrient adequacy, intakes of animal-source foods would have to be doubled, accounting for at least 27% of calories, and intakes of phytates-rich plant-sourced foods, such as whole grains, pulses and nuts, would need to be considerably reduced (Fig. 1b) 27 .
In 2019, the EAT-Lancet Commission also expressed confidence that widespread uptake of their recommended diet would reduce the incidence of non-communicable diseases and overall mortality -they estimated that approximately 11 million premature deaths among adults could be avoided annually through global adoption of the diet 3 .Zagmutt and colleagues were the first to question these estimates of avoided mortalitiesthey identified flaws in the assumptions and methods used, and their corrected analysis suggested that any mortality reduction effect of the EAT-Lancet diet was no greater than the impact of energy consumption changes that would prevent under-weight, over-weight and obesity alone 29,30 .Adherence to the EAT-Lancet reference diet did show beneficial associations for ischaemic heart disease and diabetes mellitus, but not for stroke nor mortality in the EPIC-Oxford study 31 , and was associated with reduced risks of cancer and all-cause mortality, but no association with cardiovascular events, in the UK Biobank cohort 32 .The authors of both of these UK-based studies noted that those most adherent to the EAT-Lancet diet were also most likely to follow a healthy lifestyle, and therefore residual confounding might operate.Furthermore, close adherence to the planetary health diet provided no additional protection from major cardiovascular events, cancer or mortality, in either the Prospective NutriNet-Santé cohort 33 , or the Prospective Urban Rural Epidemiology study 34 .

QUALITY AND TRANSPARENCY OF THE INPUT DATA -USE OF UNPUBLISHED DATA BY THE GBD 2019 RISK FACTORS STUDY
Compared with the GBD 2017 estimates 9 , the GBD 2019 estimates of the risks of many dietary factors 10  Whilst all previous GBD analyses, including the GBD 2017 analysis, used data from published systematic reviews and metaanalyses, the evidence for the 2019 dietary risk factor estimates came from in-house, newly conducted, systematic reviews and meta-regressions.These analyses had not been peer-reviewed nor published, and no assessments of certainty were documented.I and many scientists, including representatives of the Academy of Nutrition Sciences and the World Cancer Research Fund (WCRF), questioned the reliability of the dramatically changed 2019 estimates 22,35 , and requested that PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 36 compliant reports of the newly conducted systematic reviews be peerreviewed and published.
It is good that these requests were eventually answered through publication of the Burden of Proof (BoP) study of the health effects associated with consumption of unprocessed red meat in Nature Medicine in October 2022 11 .However, this publication raises further issues.
The relative risk curves and the conclusions of the BoP 2022 Study 11 are very different from those reported in the GBD 2019 Risk Factors Study 10 .This is particularly striking for the three cardiovascular outcomes.The GBD 2019 Study indicated that risks for ischaemic heart disease, ischaemic stroke and haemorrhagic stroke increased significantly even with moderate intakes of red meat (50 g/day), and that risk increased further with greater intakes (Fig. 2a-c).By contrast, the relative risk curves of the BoP 2022 study are either considerably flatter (ischaemic heart disease and ischaemic stroke, Fig. 2d, e), or show a trend towards protection (haemorrhagic stroke, Fig. 2f).It is noteworthy that none of these BoP relative risk curves are statistically significant, even with intakes as high as 200 g/day.The overall conclusions of the BoP 2022 study were that there is no or only very weak evidence that unprocessed red meat is associated with any increased risk.These BoP findings are in agreement with other recently conducted systematic reviews of randomised trials and cohort studies [37][38][39] .
The large disparities between the conclusions of the GBD 2019 and BoP 2022 reports concerning unprocessed red meat, highlights the importance of expert peer review, of compliance with the PRISMA statement for all newly conducted systematic reviews and metanalyses 36 , and of compliance with the GATHER (Guidelines for Accurate and Transparent Health Estimates Reporting) statement for all reports of global health estimates 40 .The large disparities also cast considerable doubt over the accuracy of the GBD 2019 estimates of the risks attributed to all other dietary factors, given that these estimates are also based on systematic reviews and meta-regressions which have not been peer-reviewed nor published.

APPROPRIATENESS OF STATISTICAL MODELLING -MONOTONIC CONSTRAINTS, RISK OF BIAS AND CERTAINTY ASSESSMENTS IN THE BURDEN OF PROOF 2022 STUDIES
As already mentioned, the GBD collaborators published a series of papers in Nature Medicine in October 2022 11,12,41,42 , which described and demonstrated their newly developed Burden of Proof methodology.Key steps of the novel modelling methodology include; 1. Estimation of the shape of the risk-outcome relationship using quadratic splines; 2. Application of monotonicity or linear-tail constraints; 3. Automatic detection and trimming of outliers; 4. Testing and correction for bias related to study design and reporting bias; 5. Computation of 95% uncertainty intervals which account for mean effects uncertainty and between-study heterogeneity; 6.Generation of the burden of proof risk function (BPRF), which estimates the level of risk closest to the null hypothesis, that is consistent with available data; and 7. Classification of each risk-outcome pair into five categories of certainty (star ratings of 1-5) based on the average magnitude of the BPRF.
Zheng and colleagues consider the existing mechanisms to quantify and rank the magnitude and uncertainty of health risks, such as the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach and Cochrane Reviews, to be largely subjective 12 .They propose that the Burden of Proof approach will provide a more consistent way to understand, evaluate and summarise evidence of risk across different riskoutcome pairs.They confirm that this methodology will inform the risk analyses of future GBD risk factor studies 12,41 .However, there appear to be a number of important flaws and limitations to this new methodology.
Step 2 mandates the default application of either monotonic or linear tail constraints.These are applied so as "to regularise spline behaviour" and "to ensure plausible risk curve behaviour at low exposure levels".However, examination of the Burden of Proof relative risk curves that describe the relationships of breast and colorectal cancer with unprocessed red meat, both with and without monotonic constraints, (Fig. 3) 11 demonstrates significant risk inflationary effects.This is particularly evident at low to moderate intakes of unprocessed red meat.For colorectal cancer, the relative risks (95% confidence interval) associated with consumption of 25 g/day of unprocessed red meat prior to, and after application of the constraint, are 1.06 (0.9-1.23) and 1.28 (1.01-1.58),respectively.For breast cancer, the equivalent relative risks are 1.12 (0.94-1.35) and 1.26 (0.98-1.56), respectively.These two to four-fold increments in risk, arising through the application of monotonic constraints, do not appear reliable nor credible.
Step 4 entails the testing and correction for bias related to study design.Table 1 illustrates the contrast between GRADE methodology, which tests for bias across seven domains [37][38][39] , and that of the Burden of Proof study of the health effects associated with unprocessed red meat, which only tests for three possible sources of bias 11 .Furthermore, less stringent levels appear to be applied -Under adjustment for prognostic factors, no risk of bias is recorded if there is adjustment for age, sex, smoking and either income or level of education (Table 1).Hence, no adjustment for family history, alcohol, diabetes mellitus, dietary quantity (energy intake or body mass index) or quality (fibre, fruit, vegetables or ultraprocessed foods intake) is required, despite considerable evidence of their associations with all six outcomes.
It is noteworthy that, even with this limited testing for bias across only three domains, the mean quality score across all cohorts and outcomes was 2.8 (quality score range; 0 [least bias] to 5 [most bias]) 11 .This high average score resulted from the majority of studies being judged to be at risk for bias because of use of a single self-report of red meat consumption, and only moderate adjustment for confounders.However, despite risks for bias being present in most studies, no bias covariates were identified as statistically relevant for inclusion in any of the six spline models 11 .This is an acknowledged limitation of the Burden of Proof methodology 12 the covariates cannot fully capture and correct for bias if all, or even a large majority, of the input studies are biased.It is of considerable concern that the end result is that no correction for bias related to study design was applied to the meta-regressions evaluating the relationships between unprocessed red meat consumption and health outcomes 11 .Finally, it is not clear how the one to five star ratings of certainty (step 7) will be utilised in future GBD risk factor studies.The conclusion of the Burden of Proof study of the health effects associated with red meat consumption was that there was either no or only very weak evidence that unprocessed red meat is associated with any increased riskonly one or two stars were awarded.Lescinsky and colleagues concluded that this level of evidence is insufficient to make any strong or conclusive recommendations.However, it appears that the GBD collaborators do intend to include such weak, uncertain and statistically insignificant evidence in future GBD estimates of deaths and DALYs -they recently commented that their "preliminary  d-f) for ischaemic heart disease, ischaemic stroke and haemorrhagic stroke, by levels of unprocessed red meat intake.Relative risks (95% confidence intervals) are plotted on the y-axis and unprocessed red meat intake (g/day) is plotted on the x-axis.The relative risk curves of the GBD 2019 Risk Factors Study (panels a-c) are for men and women aged 50-54 years, and were drawn using data from Table S7A of Supplementary Appendix 1 10 .The relative risk curves of the Burden of Proof 2022 Study (panels d-f) are for adult men and women, and were drawn using data from Supplementary Table 7 11 .
estimates for GBD 2021 remain on the order of several hundred thousand deaths attributable to red meat consumption" 43 .

POST PUBLICATION PROCESSES -DELAYED OR NON-CORRECTIONS OF ERRORS AND LIMITATIONS
Medical and scientific journals recognise the importance of postpublication commentary on published research as necessary to advancing scientific discourse.Hence, post-publication letters, commentaries and matters arising articles involving challenges or clarifications of the published work are generally welcomed.Once important errors are identified and confirmed, it is of considerable importance that appropriate corrections are rapidly published.Indeed, immediate correction of all errors of fact is mandatory according to The Lancet's own guidelines, the Committee on Publication Ethics, and the International Committee of Medical Journal Editors [44][45][46] .It is of concern that, despite limitations and errors, being publically acknowledged by the EAT-Lancet and GBD authors, respectively, no corrections have been applied to the published papers, and the estimates remain unchanged on their websites 47,48 .
As a consequence these reports continue to be extensively cited, and many research groups continue to use the uncorrected estimates of risk and optimal intakes of the GBD 2019 Risk Factors Study and the EAT-Lancet planetary health diet, in their modelling studies [5][6][7][49][50][51][52][53] .
Fadnes and colleagues estimated that changing from a typical Western diet, which they (incorrectly) claimed included 100 g/day and 50 g/day of unprocessed red meat and processed meats, respectively, to a diet which totally excludes these foods, would increase life expectancy by at least three years for both men and women 5 .Romanello and colleagues in the 2022 report of the Lancet Countdown on health and climate change reported that approximately 800,000 and 600,000 annual deaths globally were caused by diets high in red meat and processed meat respectively 6 .
Happily, recent reports from the Food and Agriculture Organisation of the United Nations and the World Health Organisation have recognised and highlighted the weaknesses and errors in these two reports 20,21 .However other agencies continue to be influenced.
The Livewell diet, which is the healthy sustainable diet recommended by the World Wildlife Fund includes even less animal-source foods than does the Eat-Lancet's planetary health diet (meat; 34 g/day versus 43 g/day, eggs; 7 g/day versus 13 g/day, seafood; 40 g/day versus 28 g/day and dairy 192 g/day versus 250 g/day) 3,58 .Given recent reports of sizable micronutrient shortfalls pertaining to the Eat-Lancet's planetary health diet [24][25][26][27] , the statement in the executive summary of the report that, "Livewell Plates are representative diets that meet national nutritional requirements," does not appear correct.
The Nordic Nutrition Recommendations (NNR) 2023 recently ranked diets high in processed meat, and diets high in unprocessed red meat, as the second and fourth highest dietary risk factors, respectively, for mortality and DALYs, in the Nordic and Baltic countries 59 .It is cause for considerable concern that the evidence for this ranking is unpublished.The paper of Clarsen and colleagues 60 was commissioned by the NNR Committee, but has been "in press" since March 2023it's estimates of the burden of dietary risk factors in the Nordic and Baltic countries between 1990 and 2021, are based on data from the also unpublished Global Burden of Diseases, Injuries, and Risk Factors Study 2021.

CONCLUSIONS
Science is the best method we have of coming to an impartial knowledge about the world 61 .In recent years there have been many calls for greater rigor, reproducibility and transparency across all the sciences [61][62][63][64][65] .In 2021 Brown and colleagues commented "Nutritional epidemiology can, and must, do better by pursuing greater scientific rigor, academic honesty, and intellectual integrity" 65 .Hence, in the conduct of systematic reviews of dietary factors, in the estimation of global health estimates, and in the use of these metrics in policy decisions and dietary guidelines, nutritional epidemiology must follow similar or analogous regulations and standards as all other scientific endeavours.In determining the optimal intakes of foods, the impacts of both nutritional deficiencies and excesses must be considered.Differences in micronutrient bioavailability from different food sources must also be recognised.PRISMAcompliant reports of all systematic reviews, and GATHERcompliant reports of all global health estimates must be published.Assessments of bias and certainty in nutrition science must be of a similar standard as those in all other health-related fields.Curve smoothing techniques cannot be allowed to inflate or create risk.Confirmed substantial errors must be immediately corrected in all paper and on-line publications, and also on institutional websites.Given the huge influence of global health estimates from the GBD Risk Factor Collaborators, and from the EAT-Lancet Commission on Food, Planet, Health, it is of even greater importance that the metrics and recommendations from these groups are rigorously and transparently evidence-based.

Fig. 2
Fig. 2 Associations between unprocessed red meat consumption and cardiovascular outcomes.Relative risk estimates from the GBD 2019 Risk Factors Study (panels a-c) and the Burden of Proof 2022 Study (panelsd-f) for ischaemic heart disease, ischaemic stroke and haemorrhagic stroke, by levels of unprocessed red meat intake.Relative risks (95% confidence intervals) are plotted on the y-axis and unprocessed red meat intake (g/day) is plotted on the x-axis.The relative risk curves of the GBD 2019 Risk Factors Study (panels a-c) are for men and women aged 50-54 years, and were drawn using data from TableS7Aof Supplementary Appendix 110 .The relative risk curves of the Burden of Proof 2022 Study (panels d-f) are for adult men and women, and were drawn using data from Supplementary Table 711.

Adequate Diet for Adults EAT-Lancet Planetary Health Diet
Quantities of foods recommended by the EAT-Lancet Planetary Health Diet and the Adequate Diet for Adults.Comparison of the percentages of calories provided by various food groups in the EAT-Lancet planetary health reference diet (panel a) 3 , and in a diet which provides sufficient micronutrients, the adequate diet for adults (panel b) 27 .
differed considerably.Disease

Table 1 .
Comparison of the GRADE and the Burden of Proof study quality and risk of bias assessments for cohort studies evaluating the relationships between unprocessed red meat consumption and adverse outcomes.